PaperLuc 1.5 related works for Semantic Foggy Scene Understanding with Synthetic Data

认真读了一遍 Semantic Foggy Scene Understanding with Synthetic Data 的 related works 部分，算是缕清了作者的一些思路，也发现了一些关于 fog detection/classify 的文章。因此粗略整理一下。

Stereoscopic Inpainting Joint Color and Depth Completion from Stereo Images

一种新的同时进行颜色和深度修复的算法。该算法将立体图像stereo images和估计的视差图disparity maps作为输入，并填充由遮挡或对象移除引入的缺失颜色和深度信息。我们首先使用基于分割的方法完成遮挡区域的差异图。完成的差异图可用于方便用户标记要移除的对象。由于一个图像中的部分被移除区域在另一个图像中是可见的，因此我们通过3D变形相互完成两个图像。最后，我们使用深度辅助纹理合成技术完成剩余的未知区域，同时填充颜色和深度。

该文章即FC（指代 Semantic Foggy Scene Understanding with Synthetic Data ）的引用68：

Our method builds on the framework of stereoscopic inpainting [68] which performs depth completion at the level of superpixels, and introduces a novel, theoretically grounded objective for the superpixel-matching optimization that is involved.

FC基于该文章完成了step 2部分的工作：

2. denoising and completion of d to produce a refined depth map d’ in meters

考虑FC的几个引用部分：

photo-consistency check

对于大多数图像完成方法，没有合适的方法来检测视觉伪像，因为算法认为结果是最佳的。通过提供附加图像并通过独立和同时对两个图像进行修复，可以通过一致性检查自动检测潜在的错误解决方案。

假设场景中的表面接近Lambertian，则可以基于相应像素的颜色一致性来检测不可靠的修复结果。

在该文章中是对完成 object removal 的 disparity map 进行检查，以检测像素点是否可靠；

FC中则对 standard stereo matching algorithm 处理后的 disparity map 和原始 stereo 图像进行检查，并将无效像素点置于M集合中。

Segmentation-based occlusion filling

采用了 segment constraint 的方法：

the disparity values vary smoothly within each segment and the corresponding 3D surface can be modeled by a plane.

主要内容为将 disparity plane 分配给每个 segment。

Our goal is to assign a disparity plane to each segment in Υ so that the disparity values of all pixels in O can be calculated using the plane parameters of the segment to which it belongs. Depending on the number of occluded pixels in a given segment, two different filling approaches are used.

两种方法分别为：

Plane fitting for (partly) visible segments

A segment S is considered as visible or partially visible if it contains enough pixels with known disparities. Criteria:

||S −O|| > max(6, λ · ||S||)

Plane assignment for the remaining segments

a greedy algorithm that works in a best-first filling order:

在FC中，类似的，首先使用SLIC划分 superpixels，再使用||S −O|| > max(6, λ · ||S||)判断是否为reliable：

These superpixels are classified into reliable and unreliable ones with respect to depth information, based on the number of pixels with missing or invalid depth that they contain.

• if reliable: 使用 Plane fitting for (partly) visible segments方法

  fit a depth plane by running RANSAC on its pixels that have a valid value for depth. (Plane fitting for (partly) visible segments)

再使用Plane assignment for the remaining segments方法:

The greedy approach(Plane assignment for the remaining segments) is used subsequently to match unreliable superpixels to reliable ones pairwise and assign the fitted depth planes of the latter to the former.

具体公式有变化，不赘述。

Stereo Processing by Semiglobal Matching and Mutual Information

本文介绍了Semiglobal Matching（SGM）立体方法。它使用基于像素，互信息（MI）的匹配成本来补偿输入图像的辐射差异。平滑约束支持像素匹配，平滑约束通常表示为全局成本函数。 SGM通过从所有方向进行路径优化来执行快速近似。讨论还涉及遮挡检测，子像素细化和多基线匹配。此外，还介绍了用于删除异常值，从结构化环境的特定问题中恢复以及间隙插值的后处理步骤。最后，提出了使用正交投影处理几乎任意大图像和视差图像融合的策略。标准立体图像的比较表明，如果考虑子像素精度，SGM是当前排名最高的算法之一，并且是最好的。复杂度与像素数和视差范围呈线性关系，这导致典型测试图像的运行时间仅为1-2秒。对基于MI的匹配成本的深入评估表明了对宽范围的辐射变换的容忍度。最后，从巨大的空中框架和推扫式图像重建的例子表明，所提出的想法在实际问题上运作良好。

来源：谷歌翻译

FC输入图片依据该文算法生成：

the available depth information in Cityscapes is not provided by a depth sensor, but it is rather an estimate of the depth resulting from the application of a semiglobal matching stereo algorithm based on [Stereo Processing by Semiglobal Matching and Mutual Information]. This depth estimate usually contains a large amount of severe artifacts and large holes, which render it inappropriate for direct use in fog simulation.

来源：FC

与我的工作重合度较小，不赘述。

Image based fog detection in vehicles

2012 IEEE Intelligent Vehicles Symposium

Classification of Images in Fog and Fog-Free Scenes for Use in Vehicles

2013 IEEE Intelligent Vehicles Symposium